Hmm...well it doesn't really have'layers' per se, as much as just a bunch of neurons. The sinusoidal/gaussian part is simply there to create a continuous property of association between different neurons. That way, since we apply a global threshold on which connections we allow (masking all others to 0), as the network adjusts the structure of the connections between its neurons - we essentially allow it to traverse what would otherwise be a binary change (which is not very conducive to gradient descent methods): ie., whether a given connection is'on' or'off'. Since this thresholded connectivity means that at any given point, many otherwise'possible' connections between various neurons will be turned'off', we are in effect able to enforce sparsity in the structure of it. So that we get fun results like in the example at the bottom where at only 3% of possible inter-neuron connections being'on', the network is continuing to improve accuracy, while still reducing the number of connections required to perform a computation cycle.